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In an adaptive population which models financial markets and distributed control, we consider 
how the dynamics depends on the diversity of the agents' initial preferences of strategies. When 
the diversity decreases, more agents tend to adapt their strategies together. This change in the 
environment results in dynamical transitions from vanishing to non-vanishing step sizes. When 
the diversity decreases further, we find a cascade of dynamical transitions for the different signal 
dimensions, supported by good agreement between simulations and theory. Besides, the signal of 
the largest step size at the steady state is likely to be the initial signal. 
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I. INTRODUCTION 

Many natural and artificial systems consist of a pop- 
ulation of agents with coupled dynamics. Through their 
mutual adaptation, they are able to exhibit interesting 
collective behavior. Although the individuals are com- 
peting to maximize their own payoffs, the system is able 
to self-organize itself to globally efficient states. Exam- 
ples can be found in economic markets and communica- 
tion networks HHI1I3. 

An important factor affecting the behavior of an adap- 
tive population is the dependence of the payoffs on the 
environment experienced by the individual agents. The 
payoffs facilitate the agents to assess the preferences of 
their decisions, hence inducing them to take certain ac- 
tions when they experience similar dynamical environ- 
ment in the future. Thus, the payoff function is crucial 
to the mechanism of adaptation. 

As a prototype of an adaptive population, the Minor- 
ity Game (MG) considers the dynamics of the buyers and 
sellers in a model of the financial market, in which the 
minority group is the winning one Q- A good indicator 
of the mutual adaptation of the agents is the reduction 
of the variance of the buyer population to values below 
those of random fluctuations |2|. Furthermore, this vari- 
ance has a universal dependence on the complexity of 
the strategies adopted by the agents, dropping to a mini- 
mum when the complexity is reduced to a universal criti- 
cal value, and rapidly rising thereafter JE |(| . Theoretical 
studies using the replica method 0,0 and the generating 
functional [UEl confirmed these general trends. 

The agents in the original version of MG uses a step 
payoff function 0, IE U meaning that the payoffs re- 
ceived by the winning group are the same, irrespective of 
the winning margin (the difference between the majority 
and minority group). Latter versions of MG uses a linear 
payoff function H U G3 , 

in which the payoffs increase 
with the winning margin. Other payoff functions yield 
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the same macroscopic behavior in their dependence of 
the population variance on the complexity of strategies 
[Tllll2j |. Thus, the behavior of the population is universal 
as long as the payoff function favors the minority group. 
A recent extension of the MG considers payoff functions 
which reward the minority agents only when they win by 
a large margin, but punish them when the winning mar- 
gin is small [Ilj ■ The extended model displays a smooth 
crossover from a minority game to a majority game when 
the payoff function is tuned. 

However, when one considers details beyond the popu- 
lation variance, one can find that the agents self-organize 
in different ways induced by different payoff functions. 
For a payoff function that favors a large winning mar- 
gin, the distribution of the buyer population is doubled- 
peaked 2J. This shows that the dynamics of the pop- 
ulation self-organizes to favor large winning margins of 
either the buyers or sellers, since the agents have adapted 
themselves to maximize their payoffs. 

In this paper, we compare the behavior of MGs using 
step and linear payoffs. Previously, we found that the 
population variance scales as a power law of the diversity 
for a step payoff [Til ITU . Diversity refers to the variance 
of the initial biases of the strategy payoffs of the agents. 
In a population with diverse preferences of strategies, the 
adaptation rate is slow, resulting in small fluctuations of 
the buyer or seller population. As we shall see, when the 
payoff function becomes linear, the scaling relation be- 
tween the variance and the diversity for the step payoff 
is replaced by a continuous dynamical transition from a 
vanishing variance at high diversity to a finite variance at 
low diversity. The dynamical transition is due to the pay- 
offs being enhanced by large winning margins at low di- 
versity. Furthermore, for systems with multi-dimensional 
signals feeding the strategies, the dynamical transition in 
each dimension do not take place at the same transition 
point. Rather, there is a cascade of dynamical transitions 
for the different signal dimensions. This rich behavior 
demonstrates the flexibility of an adaptive population for 
self-organizing to states in which agents maximize their 
payoffs, and is hence important in the modeling of eco- 
nomics and distributed control. 
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II. THE MINORITY GAME 



The Minority Game model consists a population of N 
agents competing for limited resources, N being odd Q. 
Each agent makes a decision 1 or at each time step, 
and the minority group wins. For economic markets, the 
decisions 1 and correspond to buying and selling re- 
spectively, so that the buyers can win by belonging to the 
minority group, which pushes the price down, and vice 
versa. For typical control tasks such as the distribution 
of shared resources, the decisions 1 and may represent 
two alternative resources, so that less agents utilizing a 
resource implies more abundance. The decisions of each 
agent are responses to the environment of the game, de- 
scribed by signal p.*{t) at time t, where fx*(t) = 1, ...,D. 
These responses are prescribed by strategies, which are 
binary functions mapping the D signals to decisions 1 or 
0. In this paper, we consider endogenous signals, which 
are the history of the winning bits in the most recent m 
steps. Thus, the strategies have an input dimension of 
D = 2 m , and the parameter a = D/N is referred to as 
the complexity. Before the game starts, each agent ran- 
domly picks s strategies. Out of her s strategies, each 
agent makes decisions according to the most successful 
one at each step. The success of a strategy is measured 
by its cumulative payoff, as explained below. 

Let £f (t) — ±1 when the decisions of strategy a are 
1 or respectively, responding to signal /i. Let a*(i, t) 
be the strategy adopted by agent i at time t. Then 
^(*) = J2i£a*(i\)/N is the excess demand of the game 
at time t. The payoff received by strategy a is then 



w <p(VNA(t)), where <p is the payoff function. For 
step and linear payoffs, f(x) — sgnx and \ respectively. 
(Here, we have implicitly assumed that an agent does not 
consider the impact of adopting a strategy, although the 
excess demand is only dependent on the adopted ones.) 
Let Q a (t) be the cumulative payoff of strategy a at time 
t. Then its updating dynamics is described by 




D/N 

FIG. 1: a 2 /N versus a with linear payoffs, for p=0, 0.25, 
I, 4, respectively, iV--251, s—2, 1000 samples. Inset: The 
corresponding plot for step payoffs. 
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FIG. 2: a 2 /N versus p with linear payoffs, a = 
0.002,0.01,0.02 for different m and N; Inset: a 2 /N versus 
p with step payoffs with m = 1, N = 127, 511, 2047, 8191 re- 
spectively. Dashed-dotted line: scaling prediction. For both 
payoffs, s=2, 1000 samples. 



n«(f + 1) = n a (t) - #f Wcp(VNA(t)). 



(i) 



Diversity of initial preferences of strategies is introduced 
by adding random biases 0Ji a to the cumulative payoffs of 
strategy a (a— 2, s) of agent i with respect to her first 
one. The biases are drawn from a Gaussian or binomial 
distribution with mean and variance R. The ratio p = 
R/N is referred to as the diversity. 

To monitor the mutual adaptive behavior of the popu- 
lation, we measure the variance a 2 /N of the population 
making decision 1, defined by 



^ = ^[yt t )(t)-(A" , t , )(i))f) 



(2) 



where the average is taken over time when the system 
reaches the steady state, and over the random distribu- 
tion of strategies and biases. 



III. DYNAMICAL TRANSITIONS 

As shown in Fig. 1, the dependence of the variance 
o /N on the complexity a for linear payoffs is very sim- 
ilar to that for step payoffs 0, 0] . For a above a uni- 
versal critical value a c (~ 0.3), the variance drops when 
a is reduced. The effects of introducing the diversity is 
also similar to that for step payoffs, namely, the variance 
remains unaffected when a > a c , but decreases signifi- 
cantly with the diversity when a < a c . 

However, there are differences when one goes beyond 
this general trend. As shown in Fig. 2, the variance 
curves at different values of a cross at at p — p c ~ 0.16, 
indicating the existence of a continuous phase transition 
at p c from a phase of vanishing variance at large p to a 
phase of finite variance at small p. 

This behavior is very different from that for step pay- 
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FIG. 3: The attractor dynamics when (a) | AA 1 1 is larger, (b) 
| Z\ j4° I is larger. 



oils, where the variance scales as p^ 1 and there are no 
dynamical transitions (Fig. 2 inset). The picture is con- 
firmed by analyzing the dynamics of the game for small 
to. The dynamics can be conveniently described by intro- 
ducing the D-dimensional vector A^(t) = t)/^- 
While only one of the D signals corresponds to the histor- 
ical signal p*(t) of the game, the augmentation to D com- 
ponents is necessary to describe the attractor structure of 
the game dynamics. Fig. 3 illustrates the attractor struc- 
ture in this phase space for the visualizable case of to = 1 . 
The dynamics proceeds in the direction which tends to 
reduce the magnitude of the components of A^(t) 0. 
However, the components of A^(t) overshoot, resulting 
in periodic attractors of period 2D. For m = 1, the 
attractor is described by the sequence U*(t) = 0, 1, 1,0, 
and takes the L-shape as shown in Fig. 3 [I3 • Note that 
the displacements in the two directions may not have the 
same amplitude. 

Following steps similar to those in [lj| , we find that for 
to not too large, and for convergence within time steps 
much less than \/R, 



A»(t + 1) = A»(t) - ^— V {VNA^{t))8^, {t) . (3) 

For step payoff's, Eq. 10 converges to an attractor con- 
fined in a _D-dimensional hypercube of size y/2/nR, irre- 
spective of the value of R. On the other hand, for linear 
payoffs, A^(t + 1) becomes a linear function of with 
a slope of 1 — \j2j-Kp. Hence, for p > p c = \/2tt ~ 0.16, 
the step sizes |A M A^(t) | converge to zero, whereas 
for p < p c , steps of vanishing sizes become unstable, re- 
sulting in a continuous dynamical transition at p c . 



IV. THE PHASE OF FINITE VARIANCE 

However, when p < p c , the step sizes for each of the 
D signals may not be equal. To see this, we monitor the 
variance for each of the D signals and rank them. The 
rth maximum variance is then given by 



S r = large 



(j[<(A") 2 >U=„.(t) ^ (<A">U=„. ( t))V)(4) 
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FIG. 4: S r versus p; Inset: a 2 /N versus p. Symbols: sim- 
ulation; dotted line: theory with one bifurcation; solid lines: 
theory with two bifurcations. N = 1001, m = 1, s = 2, 1000 
samples. 
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FIG. 5: S r versus p for (a) m — 2, (b) m = 3. In both cases, 
N = 1001, s = 2, 1000 samples. 



where large At (/(/Lt), r) is the rth largest function f(p) for 
p=l,...,D. 

As shown in Figs. 4-5, the step sizes for each of the 
D signals do not bifurcate simultaneously at p = p c , 
Rather, only their first maximum bifurcates from zero 
when p falls below p c , while the step sizes for the re- 
maining D-l signals remain small. When the diversity 
further decreases to around 0.05, the second maximum 
becomes unstable as well, and a further bifurcation takes 
place. For to > 2, there are further bifurcations of the 
third or higher order maxima, resulting in a cascade of 
dynamical transitions when the diversity decreases. 

This cascade of transitions is confirmed by analysis. 
For to = 1, we can generalize Eq. © to convergence times 
of the order y/R. Assuming without loss of generality 
that A 1 bifurcates while A remains small, the variance 
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FIG. 6: The number of samples with step size responding to 
signal 1 being the maximum, out of 1000 samples for N = 
1001, m — 1, and s — 2. The initial signal is 1. 



of the buyer population, as derived in [T^, is 



N 



-(AA 1 ) 2 , AA^erf. 
N 32 V 



/AA 1 



(5) 



where A A 1 is the step size responding to signal 1. As 
Fig. 4 inset shows, the analytical and simulation results 
well agree down to p ~ 0.05. However, when the diver- 
sity decreases further, this simple analysis implies that 
the variance will saturate to a constant N/32, whereas 
simulation results are clearly higher. 

This discrepancy is due to a further bifurcation of the 
minimum step size. This can be analyzed by considering 
the effect of a perturbation SA (t) in the direction of A . 
After a period of 4 steps, the accumulated perturbation 
becomes 



over samples of different initial conditions, yield the the- 
oretical curves in Fig. 4 and inset, agreeing very well with 
simulation results. Similarly, the agreement between an- 
alytical and simulation results are satisfactory for m = 2. 

Since the attractors have asymmetric responses to dif- 
ferent signals, we also study their dependence on the ini- 
tial states. Letting the system start from a certain state 
(say, state 1 for m = 1) for a given sample, Fig. 6 shows 
that the initial state is more likely to have the largest 
step size in the attractor for m — 1. Simulations show 
that higher values of m share the same trend. 



V. CONCLUSION 

We have studied the behavior of an adaptive popula- 
tion using a payoff function that increases linearly with 
the winning margin. We found a continuous dynamical 
transition when the adaptation rate of the population 
is tuned by varying their diversity of preferences. This 
is in contrast with the case of payoff functions indepen- 
dent of the winning margin, in which no phase transi- 
tions are found. Furthermore, we found a cascade of dy- 
namical transitions in the responses to different signals. 
This shows that an adaptive population has the ability to 
self-organize to globally efficient states and display a rich 
behavior, although the individual agents make selfish de- 
cisions. Hence, despite the simplicity of the population 
models, they are able to capture the essential features of 
economic markets and distributed control. 



SA°{t + A) 



:(1 



(AA 1 ) 2 

e 8 p 



,5A (t). (6) 



At p = 0.0459, where AA 1 = 0.9775, the coefficient on 
the right hand side of Eq. © reaches the value 1, and 
5A°(t) diverges on further reduction of p. Numerical it- 
erations of the analytical equations for A^(t), averaged 
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